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ABSTRACT 


This  paper  deals  with  new  and  simple  representations  of  3-D  points  in  a moving- 
observer  coordinate  system.  Assuming  rectilinear  motion  with  no  rotation  of  an 
observer  where  the  optical  axis  coincides  with  the  direction  of  motion,  and  a stationary 
scene,  points  in  3-D  space  that  lie  on  a particular  3-D  surface  produce  constant  value  of 
some  nonlinear  function  of  the  measurable  image  optical  flow.  Five  sets  of  different 
surfaces  are  introduced  and  there  is  one  optical-flow  based  constant  value  for  each  sur- 
face. We  called  these  values  "invariants".  It  is  shown  how  to  extract  these  invariants 
and  how  to  use  them  for  representing  3-D  space. 


Ill 


1.  INTRODUCTION 


During  eye  motion  in  a stationary  environment,  the  projection  of  objects  in  the 
world  is  continuously  changing,  but  we  perceive  the  world  as  stationary.  Are  there  pro- 
perties of  the  image  that  under  some  transformation  remain  constant  during  the  motion 
of  the  eye?  In  other  words,  are  there  visual  invariants? 

Gibson  [11]  capured  this  idea  as  follows: 

"If  invariants  of  the  energy  flux  at  the  receptors  of  an  organism  exist  and  if  these 
invariants  correspond  to  the  permanent  properties  of  the  environment,  and  if  they  are 
the  basis  of  the  organism’s  perception  of  the  environment  instead  of  the  sensory  data 
on  which  we  have  thought  it  based,  then  I think  there  is  new  support  for  realism  in 
epistemology  as  well  as  for  a new  theory  of  perception  in  psychology." 

This  paper  deals  with  new  and  simple  representations  of  3-D  points  in  a moving- 
observer  coordinate  system.  Assuming  rectilinear  motion  with  no  rotation  of  an 
observer  where  the  optical  axis  coincides  with  the  direction  of  motion,  and  a stationary 
scene,  points  in  3-D  space  that  lie  on  a particular  3-D  surface  produce  constant  value  of 
some  nonlinear  function  of  the  measurable  image  optical  flow.  Five  sets  of  different 
surfaces  are  introduced  and  there  is  one  optical-flow  based  constant  value  for  each  sur- 
face. We  called  these  values  "invariants".  For  example,  one  invariant  corresponds  to 
all  points  in  3-D  that  lie  at  the  same  distance  from  an  observer.  This  means  that  a non- 
linear function  of  optical  flow  will  result  in  the  same  value  for  all  points  on  a particular 
sphere  that  surrounds  the  observer. 

The  optical  flow  analysis  takes  place  in  a spherical  coordinate  system 

(rather  than  X-Y-Z).  In  this  representation,  since  we  deal  with  translation  only  along 
the  optical  axis,  any  point  in  this  coordinate  system  moves  along  a constant  <j)  radial 
line  and  can  be  processed  independently  of  any  other  point. 

There  are  several  advantages  to  using  these  invariants: 

1.  They  are  measured  in  camera  coordinates,  i.e.,  there  is  no  need  to  transform  to 
object’s  coordinate  system. 

2.  They  can  be  very  useful  when  local  qualitative  as  well  as  quantitative  vision-based 
motion  related  decisions  need  to  be  made  e.g.,  how  to  avoid  an  obstacle. 

3.  Only  one  camera  is  needed  to  extract  these  invariants,  and  all  measurements  can 
be  obtained  from  the  visual  data. 

4.  The  magnitude  of  the  camera  velocity  vector  need  not  be  known.  The  invariants 
are  measured  in  time  units  rather  than  distance  units. 
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5.  Using  the  invariants  it  is  possible  to  obtain  simple  representation  of  space,  and 
define  obstacles  in  a relatively  simple  way. 

6.  The  measurement  of  these  invariants  become  very  simple  when  logarithmic  retinas 
are  used. 

7.  These  calculated  values  of  optical  flow  can  be  used  to  find  relative  locations  of  3- 
D points  without  first  reconstructing  the  3-D  world. 

Several  uses  of  the  invariants  such  as  understanding  of  fixation,  analysis  of  loom- 
ing, robust  methods  for  3-D  reconstruction,  etc.  are  mentioned  in  the  text. 

2.  COORDINATE  SYSTEM  FOR  3-D  SPACE 

In  a rectilinear  motion  with  no  rotation,  points  in  the  image  plane  move  away 
from  the  Focus  of  Expansion  (FOE)  (Figure  la)  and  towards  the  Focus  of  Contraction 
(FOC)  (Figure  lb).  Based  on  this  observation  we  use  an  RS-O  spherical  rather  than 
an  X -Y  -Z  cartesian  representation  of  points  in  space,  which  reduces  to  a 0-C)  represen- 
tation in  the  image  domain. 

Figures  2 and  3 show  the  chosen  coordinate  system  and  the  definitions  of  r and 
the  angles  0 and  (}).  (Note  that  the  R -0-0  coordinate  system  corresponds  to  the  velocity 
egosphere  defined  in  [7].)  If  the  optical  axis  coincides  with  the  translational  vector  then, 
in  the  image  domain,  constant  (j)  corresponds  to  a radial  line  that  emerges  from  the  FOE 
and  constant  6 corresponds  to  a circle  whose  center  is  the  FOE.  Given  a point  in  carte- 
sian coordinate  system,  it  can  be  transformed  to  a (r-O-tj))  point  in  the  (/?-0-O) 
domain  and  vice  versa. 

In  this  paper  we  assume  that  the  camera  (observer)  undergoes  translational 
motion  along  the  optical  axis.  Using  the  R-Q-^  coordinate  system,  any  point  in  the 
image  domain  moves  radially  i.e.,  along  a constant  (j)  line.  In  this  coordinate  system  a 
point  in  the  image  can  be  processed  independently  of  any  other  point,  and  a constant  (\> 
line  may  be  processed  as  a 1-D  image.  For  example,  in  a translational  motion  when  the 
optical  axis  coincides  with  the  translation  vector,  each  point  on  an  edge  in  the  image 
moves  radially  away  from  the  FOE.  This  point  can  be  traced  and  both  its  optical  flow 
and  location  in  the  image  can  be  obtained  independently  of  any  other  point  in  the 
image. 

3.  MOTION  INVARIANTS 

We  describe  five  invariants.  First,  we  derive  the  general  relationship  between  the 
moving  observer  and  a point  in  3-D  space.  Based  on  this  relationship  the  invariants  are 
obtained.  It  is  shown  that  points  in  space  that  lie  on  a specific  3-D  surface  share  the 
same  time-dependent  invariant.  Geometrical  invariants  are  introduced,  followed  by  a 
derivation  of  each  invariant  from  the  image  optical  flow. 
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The  invariants  are  time-dependent  only,  i.e.,  points  in  space  are  described  in  terms 
of  time  (scaled  space)  [1,2,4]  rather  than  their  location  in  space. 

Refer  to  Figure  4.  For  a rectilinear  and  continuous  motion  of  the  camera,  any 
point  in  space  obeys: 


Al  = V At  sin0  (1) 

A/ = (r-Ar)  tanA0  (2) 

Except  for  0=0  and  0=7t,  then  from  (1)  and  (2),  for  Ar-^0,  tanA0— >A0,  and  A0Ar— >0 

0 


V =r- 


sin0 


(3) 


where  the  dot  denotes  differentiation  with  respect  to  time.  0 is  the  optical  flow  along  a 

0 

radial  line,  i.e.,  for  constant  <j).  The  value  r——  is  the  same  for  all  points  (except 

sin0 

those  with  0=0  and  0=7t)  in  3-D  space.  If,  in  addition,  the  velocity  V of  the  observer 
remains  constant,  then  this  relationship  holds  for  all  time  instants.  0 and  0 can  be 


measured/computed  at  each  instant  of  time,  and  hence  the  nonlinear  function 
be  obtained. 


0 


sin0 


can 


The  following  is  a description  of  the  five  invariants  in  terms  of  optical  flow.  We 
denote  them  by  and  They  all  have  units  of  —r — . 

Xr  Xq-  Xq  Xp  time 

1 0 

1.  The  equal  range  invariant  — = — — (Figure  5):  All  points  in  3-D  space  (except 

Xr  sin0 

those  that  lie  on  the  motion  axis,  i.e.,  with  0=0  and  0=7t)  that  lie  on  a sphere 
whose  center  is  the  pinhole  point  of  the  camera  share  this  invariant,  i.e.,  have  the 
same  Xr.  In  this  case  the  radius  r is  constant,  and  so,  using  Equation  (3),  the 
V 0 

ratio  — is  kept  constant,  and  — — remains  constant.  The  meaning  of  this  invari- 
r sin0 

0 

ant  is  that  the  modified  optical  flow  — — is  the  same  for  all  points  on  a sphere 

sin0 

(except  those  which  are  on  the  axis  motion  of  the  camera).  Points  inside  the 

0 

sphere  ("close"  points)  produce  higher  values  of  . ^ and  points  outside  the 


sin0 


sphere  ("far"  points)  produce  smaller  values  of 


0 


sin0 


Therefore  it  is  possible  to 


find  the  relative  distance  of  a point  simply  by  calculating  or  measuring  this  value. 
Points  in  3-D  space  can  be  viewed  as  lying  on  shells,  each  of  which  has  a 

different  — ^ invariant. 
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2. 


3. 


4. 


5. 


The  fixation  invariant  — =6  (Figure  6):  All  points  in  3-D  space  (except  those  that 

Xt 


lie  on  the  motion  axis,  i.e.,  with  6=0  and  6=7c)  that  lie  on  a torus  obtained  by 
rotating  a circle  which  is  tangent  to  the  direction  of  motion  axis  about  this  axis, 
share  this  invariant,  i.e.,  have  the  same  Xj.  This  circle  is  an  "Equal  Flow  Circle" 

r 

(EFC)  as  described  in  [6].  In  this  case  the  diameter  of  the  torus,  i.e.,  — — is  con- 

sin0 

stant,  and  so  using  Equation  (3),  0 remains  constant.  (Note  that  r is  the  distance 
from  the  observer  to  the  point  and  not  the  radius  of  the  torus.)  An  extension  of 
this  invariant  to  a more  general  motion  of  a camera  has  led  to  a quantitative 
approach  to  camera  fixation  [5,6]  and  to  road  following  [10]. 


1 0 

The  looming  invariant  — = — — (Figure  7):  All  points  in  3-D  space  (except 

tan0 

those  that  lie  on  the  motion  axis,  i.e.,  with  0=0  and  0=7t)  that  lie  on  a sphere 
which  lies  in  front  of  the  camera  share  this  invariant,  i.e.,  have  the  same  X5.  The 
center  of  the  sphere  lies  on  the  optical  axis  of  the  camera,  and  the  camera  lies  on 


the  sphere’s  surface.  In  this  case  the  diameter  of  the  sphere 


COS0 


is  constant,  and 


SO,  using  Equation  (3), 


6 


tan0 


remains  constant.  It  has  been  shown  by  Raviv  in  [9] 


that  aU  points  on  a particular  sphere  result  in  the  same  visual  looming. 


The  clearance  invariant  = — — (Figure  8):  All  points  in  3-D  space  (except 

'^c  sin'^O 

those  that  lie  on  the  motion  axis,  i.e.,  with  0=0  and  0=7t)  that  lie  on  a cylindrical 
surface  whose  axis  coincides  with  the  camera  translational  motion  vector  share 
this  invariant,  i.e.,  have  the  same  x^ . In  this  case  the  radius  of  the  cylinder  r sin0 

0 ... 

is  constant,  and  so,  using  Equation  (3),  — — remains  constant.  This  mvanant 

sin26 

has  been  used  by  Raviv  [3]  to  develop  a robust,  integration-based,  and  massively 
parallel  method  for  reconstructing  3-D  scenes.  Albus  [7]  has  suggested  that  this 
invariant  can  be  used  to  measure  clearance. 

1 26 

The  time  to  contact  invariant  — = — (Figure  9):  All  points  in  3-D  space 

Xp  sin20 

(except  those  that  lie  on  the  motion  axis,  i.e.,  with  0=0  and  6=7c)  that  lie  on  a 
plane  which  is  perpendicular  to  the  direction  of  motion  of  the  camera,  share  this 
invariant,  i.e.,  have  the  same  Xp.  In  this  case  rcos0  (the  distance  from  the  sur- 

20 

face)  is  constant,  and  so,  using  Equation  (3),  ■ remains  constant. 

sin26 
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Xp  is  the  "time  to  contact"  as  described  by  Lee  in  [1],  and  by  Lee  and  Reddish 
[2].  Lee  observed  that  two  macroscopic  visual  parameters  are  essential  for  animals’ 

d X 

behavior,  the  time-to-contact  x and  its  derivative  . He  also  showed  how  to  derive 

dt 

these  parameters  from  the  optical  flow.  These  parameters  "tell"  the  animal  how  to  con- 
trol its  motion  in  order  to  avoid  collision.  Diving  birds  use  it  to  fold  wings  for  entry 
into  water  [2].  x is  specific  to  the  immediacy  of  contact  (i.e.,  when  contact  will  be 
dx  . 

made),  and  is  specific  to  the  harshness  of  contact  (i.e.,  the  type  of  contact)  with  the 

environmental  surface.  In  this  paper  we  referred  to  the  "time  to  contact"  as  Xp  and 
showed  its  formulation  in  an  image  centered  spherical  coordinate  system.  More  about 
extraction  of  Xp  from  visual  data  can  be  found  in  [3]. 

Figure  10  is  a summary  of  the  invariants.  It  shows  the  basic  relationship  between 
space,  speed  and  optical  flow  (top  equation),  from  which  the  five  invariants  are  derived. 
Based  on  geometrical  properties,  the  time-dependent  invariants  are  shown  as  a function 
of  optical  flow.  The  geometrical  interpretations  of  all  invariants  are  summarized  at  the 
bottom  of  Figure  10. 

4.  REPRESENTATIONS  USING  MOTION  INVARIANTS 

Two  different  representations  of  3-D  space  using  the  invariants  are  shown  in  Fig- 

1 0 

ure  11.  Figure  11a  shows  a representation  that  uses  the  clearance  invariant  — = — ^ 

'Cc  sin^e 

1 20 

(Figure  8)  and  time  to  contact  invariant  — = — — (Figure  9).  Figure  lib  shows  a 

Xp  sin20 

representation  that  uses  the  fixation  invariant  -^=0  (Figure  6)  and  the  looming  invari- 

Xj 

1 0 

ant  — = (Figure  7).  In  both  11a  and  11b  Figures,  the  third  dimension  is  O. 

X5  tan0 
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Figure  la;  Optical  Bow  Relative  to  the  Focus  of  Expansion  (FOE) 


7 


Figure  lb:  Optical  Flow  Relative  to  the  Focus  of  Contraction  (FOC) 
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Figure  2:  The  Coordinate  System 
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Figure  3:  Tlie  0—0  Image  Domain 
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Figure  5:  Meaning  of  the  — — 
(a)  In  2-D 


Invariant 

sin0 


(b)  In  3-D:  Equal  distance  sphere 
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(b) 

Figure  6:  Meaning  of  the  = 6 Invariant 
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(a)  In  2-D 

(b)  In  3-D:  Equal  flow  toms 
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sphere 


Figure  7:  Meaning  of  the  ^ Invariant 

7^  tan0 

(a)  In  2-D 

(b)  In  3-D:  Equal  looming  sphere 
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Figure  8:  Meaning  of  the  = — — Invariant 

Tc  sin^0 


(a)  In  2-D 

(b)  In  3-D:  Equal  clearance  cylinder 
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(b)  In  3-D:  Equal  depth  plane 
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Figure  10:  Summary  of  the  Invariants 
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Figure  11b:  Representation  2 
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